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1.  INTRODUCTION 

The  title  of  the  contract  for  which  this  is  a  final  report  is 
"Development  of  a  Nonlinear  Programming  System."  More  fundamentally, 
the  research  is  concerned  with  factorable  functions,  functions  which  are 
complicated  compositions  of  transformed  sums  and  products  of  transformed 
sums  and  products  ...  of  functions  of  a  single  variable. 

For-  the  past  twenty  years  considerable  effort  has  been  expended 
on  the  development  of  nonlinear  programming  theory  and  algorithms  for 
solving  nonlinear  programming  problems.  Some  of  these  algorithms  have 
been  implemented  on  digital  computers.  It  is  fair  to  say,  however,  that 


solving  a  complicated  nonlinear  programming  problem  by  a  computerized 


nonlinear  programming  algorithm  is  not  an  automatic  process.  Unlike 
linear  programming,  where  computerized  algorithms  (variations  on  the 
simplex  method,  usually)  have  long  been  able  to  solve  problems  of  large 
size,  nonlinear  programming  is  still  in  its  infancy  as  regards  its 
ability  to  guarantee  solutions  to  problems  of  even  moderate  size. 


Serial  18218 


2 


6  June  1983 


A  major  barrier  to  solving  nonlinear  programming  problems  is  the 
lack  of  a  computationally  oriented  way  of  representing  nonlinear  func¬ 
tions  of  several  variables.  Examples  of  real  world  problems  which  give 
rise  to  complicated  nonlinear  functions  are  weapon/ target  allocation, 
optimal  inventory  policy  determination,  minimum  weight  structural  design, 
and  parameter  estimation  through  nonlinear  least  squares  regression  and 
maximum  likelihood  estimation. 

Algorithms  for  solving  problems  such  as  those  above  are  often  not 
as  efficient  as  they  could  be  because  of  the  inability  to  compute  auto¬ 
matically  quantities  related  to  the  complicated  nonlinear  functional  re¬ 
lationships  that  describe  the  models.  For  example,  the  accurate  and 
speedy  computation  of  first  derivatives  is  a  usual  requirement  for  algo¬ 
rithms  which  solve  systems  of  nonlinear  equations. 

y  The  primary  objective  of  this  project  was  to  provide  a  language 
that  is  user-oriented  and  which  establishes  an  interface  between  optimi¬ 
zation  problems  stated  algebraically  and  computer  coded  algorithms  for 
solving  them. 

The  basis  for  this  language  is  a  new  approach  for  representing, 

nonlinear  functions  of  several,  variables.  The  class  of  functions  to 

T  9^ 

which  this  approach  applies  is  called  factorable  functions.  The 
basic  definition  is  ns  follows: 

a  function  f(x,,...,x  )  is  a  t'arki'i'.ibi e  funaLion  of  several 
1  n 

variables  if  it  can  be  represented  as  the  last  in  a  finite 

sequence  of  functions  { f  }  that  are  composed  as  follows: 


fj(x)  =■■  X.  ,  for  j 


,  n 
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For  j  >  n 


or 


f  (x)  is  either  o£  the  form 
fR(x)  +  f(x>£  >  k’^  <  J  » 
fk(x)  •  f(x)^  ,  k,£  <  j  , 

T[£k(x)]  ,  k  <  j  , 


where  T(f)  is  a  function  of  a  single  variable. 

A  brief  general  description  of  the  factorable  programming  ap¬ 
proach  is  given  in  [McCormick  1979],  where  its  relevance  to  problems  in 
engineering  design  is  emphasized.  A  more  general  description  is  con¬ 
tained  in  Chapter  3  of  the  recent  textbook  [McCormick  1983].  Details  of 
the  various  developments  contained  in  the  papers  in  the  cumulative  bibli¬ 
ography  at  the  end  of  this  report  have  been  touched  on  in  previous 
progress  reports.  Here  just  a  brief  summary  of  the  work  accomplished 
will  be  given.  In  the  fall  of  1983  a  book  giving  the  integrated  details 
of  this  and  current  work  will  be  begun  by  McCormick  while  he  is  on 
sabbatical,  leave. 


2.  COMPUTER  CODES  AND  APPLICATIONS  PAPERS 

The  first  computer  code  implementing  the  concept  of  factorable 
functions  was  written  by  Robert  Pugh  and  two  of  his  associates.  This 
was  described  in  the  publication  [Pugh  1972].  The  automatic  computation 
of  first  and  second  derivatives  and  the  dyadic  structure  of  the  Hessian 
matrices  of  factorable  functions  was  an  important  feature  of  this  code. 

The  current  version  of  the  program  implementing  the  factorable 
function  approach  to  solving  problems  is  documented  in  [Chaemi  and 


Copy  available  h 
peimit  1  Fy  i  C 


PT1C  does  not 
j  p  eduction 
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McCormick  1979].  Publications  relevant  to  this  are  [Chaemi  1979] s 
[Monaco  1979],  and  [Ghaemi  and  McCormick  1980]. 

Two  user  oriented  applications  papers  which  show  examples  of 
optimization  problems  and  how  they  could  be  encoded  in  the  factorable 
way  are  [McCormick  1975]  and  [McCormick  1977]. 

3.  SENSITIVITY  ANALYSTS 

Automatic  computation  of  first  order  post  optimality  sensitivity 
analysis  based  on  factorable  functions  was  pursued  in  the  dissertation 
[DeSilva  1978]  and  in  the  paper  [DeSilva  and  McCormick  1978],  The 
capability  of  doing  this  sensitivity  computation  automatically  was 
implemented  in  the  code  described  in  [Chaemi  and  McCormick  1979], 

4.  USE  IN  MATRIX  METHODS 

The  implications  of  the  factorable  representation  of  nonlinear 
functions  for  matrix  methods  used  in  nonlinear  programming  algorithms 
is  great.  -Some  numerical  techniques  based  on  the  outer  product  repre¬ 
sentation  of  the  Hessian  matrix  of  factorable  functions  have  been  de¬ 
veloped.  For  the  unconstrained  case  a  stable  method  was  developed  in 
the  dissertation  [Emami  1978]  and  is  described  also  in  [Hmami  and 
McCormick  1978]. 

Numerical  techniques  implementing  algorithms  for  the  linearly 
constrained  problem  were  developed  in  the  dissertation  [Chotb  1980], 


i 
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5.  ALGORITHM  COMPARISON 

The  factorable  approach  provides  a  new  way  to  evaluate  competing 
algorithms  in  a  rigorous  way,  particularly  by  providing  a  point  of  view 
to  make  comparable  the  amount  of  work  required  to  calculate  function 
values,  first  and  second  derivatives.  A  start  in  this  area  is  in  the 
dissertation  [Shayan  1979],  with  draft  papers  [McCormick  and  Shayan, 
1981a,  1981b]. 

6.  COMPUTATION  OF  EIGENVECTORS  AND  EIGENVALUES 

The  natural  form  of  Hessian  matrices,  that  is,  the  dyadic  form, 
has  implications  for  the  computation  of  eigenvectors  and  eigenvalues. 
Some  investigations  into  this  are  in  [Falk,  ct  at.,  1974]  and  in 
[McCormick  1983,  Chapter  2], 

7.  OBTAINING  AND  VERIFYING  GLOBAL  SOLUTIONS  FOR  NONLINEAR  OPTIMIZATION 

The  simple  structure  of  factorable  functions  allows  for  two 
computations  that  have  implications  for  nonconvex  programming  problems 
(i.e.,  problems  for  which  local  solutions  may  exist,  which  are  not 
global  solutions).  One  step  in  the  determination  of  a  global  solution 
is  the  ability  to  form  convex  programming  problems  that  underestimate 
the  original  one.  The  use  of  factorable  programming  problems  in  this 
regard  is  in  [McCormick  1 976 1  .  The  second  step  is  the  abil  ity  to 
verify  that  a  local  miniinizer  is  a  global  minimizer  in  a  hyperrec tangl c 
This  is  discussed  for  the  unconstrained  case  in  [Mane. ini  and  McCormick 
1  976]  and  [Mancini  and  McCormick  1 9 7 9 [ .  For  the  general  nonlincarly 
constrained  case  tin's  is  discussed  in  Section  18.3  of  [McCormick  1983]. 
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For  functions  of  one  variable  the  materia]  in  [McCormick  1981] 
combined  with  the  developments  in  [McCormick  and  Shnyan  1981  |  and 
[Shayan  1979]  present  an  interesting  wav  of  obtaining  a  global  solution. 

Global  solutions  can  also  be  obtained  by  converting  factorable 
problems  to  equivalent  separable  problems  When  the  functions  involved 
are  factorable,  this  can  be  an  automatic  procedure  (see  [McCormick  1972, 


1981])  . 
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